The Troubling Aspects of a Building Block Hypothesis for Genetic Programming

In this paper we rigorously formulate the Schema Theorem for Genetic Programming (GP). This involves defining a schema, schema order, and defining length and accounting for the variable length and the non-homologous nature of GP's representation. The GP Schema Theorem and the related notion of a GP Building Block are used to constlUct a testable hypothetical account of how GP' searches by hierarchically combining building blocks. Since building blocks need to have consistent above average fitness and compactness, and since the term in the GP Schema Theorem that expresses compactness is a random variable, the proposed account of GP search behavior is based on empirically questionable statistical assumptions. In particular, low Valiance in schema fitness is questionable because the performance of a schema depends in a highly sensitive manner on the context provided by the prograInS in which it is found. GP crossover is likely to change this context from one generation to the next which results in high variance in observed schema fitness. Low variance in compactness seems fortuitous rather than assured in GP because schema-containing prograInS change their sizes essentially at random. Introduction Based upon the Schema Theorem [Holland 1975], the Building Block Hypothesis [Holland'92, Go1dberg'89] is the original account of the search power in Genetic AlgOlithms (GAs). Holland's work is fundaInenta1 analysis, and from it more precise or clarified explanations (some diverging from a schema-based approach) of GA search behavior have been pursued. Some experimental [Mitchell, FOlTest et al. 1991; FOtTest and Mitchell 1992] and theoretical [Grefenstette and Baker 1989; Radcliffe 1991; Altenberg 1994] research has even diminished the value of the Building Block Hypothesis as a description of how GAs search or the source of the GA's power. For eXaInple, it is unclear which classes of search functions (even those with building block sllucture) the GA will solve faster than other seal'ch techniques such as hill clinIbing [Mitchell, Holland et al. 1993]. The Schema Theorem and Building Block Hypothesis were but the start of the process of detailed analysis of GAs. Genetic Programming (GP) [Koza 1992] is "a (relatively) new kid on the evolution-based algorithms block". The enthusiasm to apply GP has outpaced the attention paid to explaining it as a seal'Ch technique. Koza [Koza 1992] presents but a brief sketchy analogy with the GA Schema Theorem and Building Block Hypothesis. Another approach based upon population genetics analysis [Altenberg 1994] has great potential however the assumptions of the simplified "generation 0" model raise the SaIne unresolved clUcia1 issues we shall present here. In this paper we go through the exercise of rigorously formulating the Schema Theorem for GP. To do so involves defining a schema, schema order, and defining length and accounting